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The  aelection  an>;-  use  of  tests  is  a  problem  which  is  con- 
stantly before  moat  guidance  workers  today.  There  is  a  constant 
search  for  new  and  better  testa  and  for  new  or  better  meana  of 
using  the  tests  available.  It  would,  of  course,  be  Impossible 
for  all  schools  to  give  a  comprehensive  battery  of  aptitude 
tests,  but  it  seems  desirable  to  use  in  the  testing  program  a 
test  which  is  an  Indicator  of  mechanical  aptitude. 

The  Revised  Minnesota  Paper  Form  Board  Teat  by  Likort  and 
Quasha  is  recognized  as  one  of  the  beat  of  its  kind  for  the 
measurement  of  apace  relationship  perception,  one  of  the  measur- 
able aspects  of  mechanical  ability.  Bingham  (2)  states, 

Ability  not  only  to  perceive  the  spatial  rela- 
tions of  objects  but  to  think  correctly  about  these 
relations  is  obviously  an  important  factor  in  mechan- 
ical aptitude.  Indeed,  it  is  so  essential  for  many 
kinds  of  work  that  numerous  blanks  and  forms  have  been 
developed  for  uae  in  measuring  this  kind  of  ability. 
One  of  the  ;aost  convenient  of  these  is  the  geometrical 
conatruction  test  known  as  the  Minnesota  Paper  rorm 
Board  recently  improved  by  Likert  and  Quasha.  This  is 
essentially  a  test  of  speed  in  recognition  of  forms 
and  space  relations.  It  does  not  measure  accurately 
the  level  of  difficulty  of  the  apace  problems  which  a 
person  is  able  to  solve. 

Since  this  test  when  given  according  to  directions  is  a 

time-limit  test  and  ao  measures  speed  of  recognition  of  forms 

and  apace  relations  rather  than  the  level  of  difficulty  of  the 

space  problems  which  a  person  can  solve,  and  since,  accord! 

to  a  statement  by  the  author a  of  the  test  in  the  manual  of 

directions,  the  problems  are  arranged  In  order  of  difficulty. 
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the  problem  arises  as  to  whether  or  not  the  predictive  value 
of  the  test  could  be  increased  by  doubling  the  time-limit, 
thus  making  it  essentially  a  work-limit  test  and  assuring  that 
all  subjects  would  reach  the  more  difficult  problems.  This 
problem  was  further  brought  to  the  attention  of  the  writer 
when  he  had  given  the  test  to  a  group  of  tenth  grade  geometry 
students,  it  was  observed  that  many  who  made  hi     adea  in 
geometry  made  low  scores  on  the  test  but  that  they  had  solved 
all  or  nearly  all  of  the  examples  correctly  as  far  as  they 
were  able  to  work  in  the  twenty-minute  tine  limit*  This  obser- 
vation and  the  foregoing  statement  by  Bingham  lead  to  the 
undertaking  of  the  solution  of  this  problem;  namely,  will  the 
work-limit  scores  or  the  time-limit  scores  on  the  Revised 
Minnesota  Paper  Form  Board  Test  give  higher  correlations  with 
shop,  mechanical  drawing,  and  mathematics  grades,  and  hence  be 
the  better  predictor  of  success  in  mechanical  and  engineering 
pursuits? 

Work-limit  as  used  in  this  study  refers  to  the  practice 
of  allowing  sufficient  time  for  all  subjects  to  complete  the 
tost;  thus  the  scores  are  measures  of  level  of  accomplishment 
and  not  a  measure  of  rate  of  recognition  of  correct  spatial 
relationships.  This  definition  is  given  because  some  authors 
have  used  the  term  "work-limit "  to  indicate  the  amount  of  time 
necessary  to  complete  a  given  number  of  tasks  or  problems. 
Using  this  definition,  Peterson  a^d  Tinker  (12)  found  that  there 
was  no  significant  difference  in  time-limit  and  work-limit 
scores  on  the  Chapman-Cook  Speed  of  Reading  Test.  However, 


the  Intent  of  this  test  la  to  aaaaure  speed,  so  obviously  the 

time  element  is  the  essential  factor  and  must  enter  into  the 

score  In  one  wa^  or  another.  Whether  speed  of  recognition  or 

level  of  accomplishment  on  the  Minnesota  Paper  Form  Board  Teat 

will  correlate  higher  with  grades  in  subjects  related  to 

engineering  and  mechanical  work  became  the  purpoae  of  thia 

study.  Authors  are  not  in  agreement  on  the  question  of  whether 

or  not  to  time  mental  tests.  The  test  used  in  this  study,  of 

course,  is  intended  to  measure  onl^  certain  mental  aspects  of 

mechanical  aptitude.  Whipple  (15)  states, 

It  is  true  that  In  many  instances  it  seema  most 
obvious  and  is  extremely  tempting  to  compare  the  work 
of  different  S's  In  terms  of  speed.   3ut  in  the 
author1 8  opinion  we  have  been  led  astray  by  thia 
temptation.  Certainly  if  we  seek  to  evaluate  the 
'higher1  mental  functions,  speed  is  not  the  primary 
index  of  efficiency,  as  Is  borne  out  by  the  evidence 
that  speed  and  intelligence  are  not  very  highly  cor- 
related. ...  As  a  general  rule  It  may  be  said  that 

■  measurements  become  more  alfnlr leant  and  reliable 
in  proportion  as  the  task  becomes  more  mechanical  and 
leas  intellectual. 

Another  rather  obvious  fact  regarding  time-limit  score8, 

yet  one  which  is  frequently  overlooked  is  further  emphasized 

by  Whipple;  namely,  that  a  time-limit  test  always  meaaurea  the 

purely  motor  aspect  of  time  taken  in  marking  the  answera  aa 

well  as  the  time  taken  in  the  mental  processes  of  arriving  at 

the  correct  answer.   It  would  be  difficult  to  eatimate  the  per 

cent  of  the  time  spent  In  deciding  which  ia  the  correct  anawer 

as  compared  with  the  time  spent  In  marking  It  In  the  proper 

place;  but  if  the  teat  is  timed,  both  the  mental  procesaea 

involved  in  determining  the  correct  answer  and  the  purely 


mechanical  process  of  recording  it  become  components  of  the 
score.  Of  course,  this  purely  mechanical  action  may  oe  a 
definite  part  of  mechanical  aptitude,  but  it  should  be  meas- 
ured by  a  separate  test  intended  to  measure  that  phase  and 
should  not  be  a  part  of  the  score  on  a  test  intended  to  meas- 
ure only  mental  processes.  Also  time  3pent  by  a  subject  in 
such  irrelevant  acts  as  blowing  the  nose,  rubbing  the  eyes, 
sharpening  or  exchanging  pencils  or  adjustin,  the  lead  if 
using  an  eversharp  and  a  multitude  of  other  such  acts  con- 
stitutes a  factor  In  the  score.  In  a  time-limit  test  there 
Is  also  the  possibility  of  errors  in  timing  and  this  possi- 
bility increases  as  the  test  becomes  more  arid  more  widely 
used  and  quite  often  by  inexperienced  and  inadequately  trained 
examiners.  Errors  in  timing,   of  cours  o,  tend  to  make  the 
scores  less  reliable. 

The  elimination  of  the  time  element  in  mental  tests, 
however,  is  not  accepted  by  many  writers.  Thurstone  (14) 
states, 

Th»re  Is  a  general  prejudice  a(  ainst  time-iimit 
examinations  based  on  the  assumption  that  one  cannot 
do  one»s  best  when  :  orced  to  won  against  time,  'te 
gave  a  series  of  examinations  both  with  and  without 
the  pressure  of  working  against  time,  and  found  that 
in  all  the  examinations  the  diagnostic  value  was  con- 
siderably increased  by  riving  the  tests  on  the  time- 
limit  basis.  It  is  true  that  our  best  work  is  net 
done  to  the  time  of  a  stop-watch  rjut  it  is  also  true 
that,  otter  things  being  equal,  the  brightest  minds 
can  not  only  work  better  but  also  faster  than  the  less 

,ted  minds.  Why  should  we  then  not  avail  ourselves 
of  one  of  tho  fundamental  differences  between  bright- 
ness and  dullness,  namely,  the  difference  in  normal 
speed  of  mental  work?  It  has  been  established  with 
considerable  certainty  that  an  examine tion  is  more 


diagnostic  of  tho  aoility  meaaurod  if  we  take  into 
consideration  tho  speed  with  which  the  student  works. 

Since  there  seems  to  be  such  disagreement  as  to  the  value 
of  timing  and  since  there  have  been  few  studies  to  confirm 
either  viewpoint,  the  need  for  such  investigation  concerning 
specific  tests  is  evident.   It  was  not  the  purpose  of  this 
study  to  arrive  at  an  Inclusive  conclusion  as  to  the  merits  of 
time-limit  and  work-limit  methods  in  administering  mental  tests 
in  general,  but  simply  to  ascertain  which  method  fives  higher 
correlations  between  this  particular  test,  The  Revised  Minnesota 
Paper  i orm  Board,  and  school  subjects  related  to  engineering 
ar:d  mechanical  pursuits. 

Tho  major  emphasis  in  this  study  was  devoted  to  the  use 
of  the  test  in  the  guidance  of  secondary  school  students.  How- 
ever, since  the  test  is  also  intended  j or  use  at  the  college 
level,  and  since  data  for  some  limited  college  groups  were 
available,  a  short  summary  of  the  results  obtained  from  them  Is 
also  presented. 
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II.  :duhk 

In  order  to  facilitate  scoring  and  to  make  It  poaalble 
to  administer  the  test  to  the  same  group   on  both  the  twenty- 
minute  and  the  i or ty -minute  time  limits,  the  Perfo-Score  Answer 
Sheets  by  J.  C.  Peterson  and  H.  J.  Peterson  were  used  instead 
of  the  method  of  answering  on  the  test  booklets  as  provided  in 
the  instructions  accompanying  the  testa.  Answering  on  the 
Perfo-Score  Sheets  instead  of  on  the  test  booklets  should  in 
no  way  affect  the  scores,  since  in  both  cases  the  answer  is 
registered  some  little  distance  from  the  example  being  solved, 
and  encircling  the  correct  letter  on  the  Perfo-Score  Sheet  is 
a  task  comparable  to  the  printing  of  the  letter  of  the  correct 
answer  in  the  proper  square  on  the  test  booklet.   In  regard  to 
the  use  of  the  separate  answer  sheet,  Dunlap  (4)  found  that  the 
use  of  the  separate  answer  sheet  is  entirely  satisfactory  espe- 
cially when  the  test  is  short  enough  to  enable  all  the  answers 
to  be  recorded  on  one  side  of  the  sheet.  Since  this  test  con- 
sists of  64  problems  and  the  Perfo-Score  Sheet  used  provides 
for  100  answers,  it  is  entirely  satisfactory  for  answering  and 
aleo  provides  facilities  for  answering  the  sample  problems  at 
the  begl  ,  in,  of  the  test. 

In  order  to  make  it  possible  to  administer  this  test  to 
the  same  group  on  both  the  standard  twenty-minute  time  limit 
and  on  the  double-time  basis  it  was  necessary  to  revls 
instructions  in  accordance  with  a  statement  which  follows 


shortly  and  also  to  use  the  answer  sheet  In  the  following  man- 
ner. A  carbon  sheet  was  attached  to  the  back  of  the  answer 
sheet  with  the  carbon  face  towards  the  answer  sheet  and  a  strip 
of  light  cardboard  was  attached  tc  provide  a  solid  backing  for 
the  answer  sheet.  Thus,  when  an  answer  was  marked  on  the  front 
of  the  answer  sheet,  a  carbon  impression  of  the  circle  was  also 
made  on  the  back  of  the  sheet.  After  the  first  20  minutes  had 
elapsed,  the  carbons  were  removed  and  the  subjects  continued 
working  for  another  twenty-minute  period.   Thus,  the  carbon 
Impressions  on  the  back  of  the  answer  sheets  provided  the  basis 
for  counting  the  scores  on  the  twenty -minute  limit  and  the 
answers  on  the  iront  of  the  sheets  gave  the  scores  en  the  work 
limit  basis.  A  subject  was  allowed  to  go  back  over  those  done 
during  the  first  20  minutes.   ;ven  though  a  subject  did  not 
attempt  the  items  consecutively  during  the  first  20  minutes, 
the  carbon  impressions  on  the  back  would  indicate  his  score 
on  the  twenty-minute  limit  and  the  penciled  circles  on  the 
front  would  indicate  his  double-time  score. 

The  question  may  arise  as  to  whether  or  not  this  method 
will  five  scores  on  the  work-limit  basis  which  are  the  same 
as  would  be  made  if  the  subjects  were  allowed  to  work  from 
the  beginning  without  being  under  the  pressure  of  time  for 
the  first  20  minutes.   It  is  acknowledged  that  it  mi^ht  not. 
However,  since  the  subjects  were  allowed  to  omit  problems 
during  the  first  20  minutes,  and  were  allowed  to  answer  them 
later  or  to  change  any  which  they  thought  had  been  answered 
incorrectly  during  the  first  20  minutes,  it  is  logical  to 
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assume  that  the  scores  made  on  the  double-tlrae  basis  do  con- 
stitute actual  work-limit  scores.  Then  there  is  the  added 
advantage  in  using  this  method  of  having  the  same  < roup  for 
correlations  thus  eliminating  the  sources  of  error  that  always 
exist  due  to  unaccountable  and  immeasurable  differences  oetween 
experimental  and  control  groups. 

The  instructions  for  administering  the  teat  under  these 
conditions  were  revised  to  read  as  follows  i 

Examiner:  Before  distributing  test  booklets  say: 

"Do  net  write  anything  on  this  test  booklet,   (Hold 
one  up,)  There  is  a  aepar&te  answer  sheet  on  which 
your  name,  the  name  of  the  test,  and  all  the  answers 
are  to  be  recorded.  When  you  receive  the  test  book- 
let, let  it  lie  on  your  desk   with  page  1  on  top, 
but  do  not  read  it  until  you  are  told  to  begin," 
Examiner:  Distribute  test  booklets;  then  say: 

"Take  your  answer  sheet  out  of  the  booklet  and  print 
your  name  and  that  of  your  school  and  your  class  in 
the  proper  spaces  on  it.   On  the  line  labeled  'subject' 
write  »Forrn  Board,  Series  AA  (or  'Series  3B»  if  that 
is  the  form  in  use ) , " 

(After  sufficient  pause  for  writing  on  answer 
sheet,  say:) 

"You  will  notice  a  sheet  of  carbon  paper  attached 
to  your  answer  sheet.  This  is  simply  to  make  a 
double  record  of  your  marks,  one  on  the  front  and 
one  on  the  back  of  your  answer  sheet.  Leave  it 
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attached  just  as  It  is  while  marking  your  answers." 

m   look  at  the  instructions  on  page  1  and  read 
them  silently  while  I  read  them  aloud." 

.  ter  reading  through  to  end  of  problem  4  aay:) 
or  practice  we  will  now  record  the  answers  to 
the  sample  proolema  on  page  1  of  the  test  booklet 
on  lines  91  to  98  of  the  answer  sheet,  since  these 
lines  are  not  used  in  the  actual  test.  The  answer 
to  problem  1  is  •■»«   Pleese  record  this  answer 
now  on  your  answer  sheet  by  drawing  a  circle  around 
the  »E»  on  line  91  of  the  answer  sheet.  The  answer 
to  problem  2  is  'A*.  Kecord  this  answer  now  by 
drawing  a  circle  around  'A*  on  line  92  of  your 
answer  sheet." 

Examiner t  Continue  this  procedure  through  problem  4; 

then  say; 

"v;hat  is  the  correct  answer  to  problem  5?" 
(When  replies  agree  say:) 

"Yes,  'C1  is  the  correct  answer.  Please  record  it 
by  encircling  the  letter  ,c»  on  line  9b  of  the 
practice  section  of  your  answer  sheet." 

Examiner:  Follow  the  same  procedure  through  questions 

6,  7,  and  8;  then  say: 

"Are  there  any  questions?  Ko  questions  will  be 
answered  after  you  have  started.   Is  there  anything 
that  any  of  you  do  not  understand?" 
(Beinf  sure  that  none  of  the  subjects  open  the  test 
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booklets  until  told  to  do  so,  continue  reading  the 
directions  starting  with:   "Some  of  the  problems." 
Omit  "Print  with  capital  letters  only"  and  "Make 
them  00  that  anyone  can  read  them"  or  in  ieate  that 
these  statements  do  not  apply.  When  you  have  fin- 
ished reading  the  directions,  show  the  group  how  the 
booklet  opens,  and  show  them  that  to  .  et  to  page  3 
the  entire  booklet  must  be  opened.   It  is  very  impor- 
tant to  show  them  how  this  Is  done.  Show  them  that 
there  are  64  problems  in  all .  Then  say : ) 
"Remember,  all  answers  are  to  be  recorded  on  the 
answer  sheet.  Record  the  answer  on  the  .  irst  line 
of  jo  ;r  answer  sheet  by  drawing  a  circle  around  the 
letter  which  precedes  \our  chosen  answer.  Record 
the  answer  to  the  second  problem  on  the  second  line 
of  your  answer  sheet  in  the  same  way  and  continue 
to  record  the  answer  to  each  problem  on  the  corres- 
ponding line  of  the  answer  sheet.   Be  careful  to 
record  each,  answer  on  the  right  line." 
"If  you  make  an  error  in  marking,  simply  mark  an  X 
through  the  circle  you  have  made,  and  then  encircle 
the  one  intended." 
"Are  there  any  questions?" 
(If  rone,  say:) 

"Open  your  booklets  and  start.  Ready?  Co." 
Examiner:  Exactly  20  minutes  from  that  time  say: 

'Stop.  y.ou  have  now  worked  for  exactly  20  minutes. 


it  •  ■ 
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Now  please  remove  the  oarbon  sheet  i  roxn  your  answer 
sheet  and  clip  the  answer  sheet  back  onto  the  card- 
board .  " 

(After  sufficient  time,  sayj) 

"The  carbon  circles  on  the  .jacks  of  your  answer 
sheets  represent  your  answers  on  a  20  minute  basis, 
Vie  are  now  .  oln  to  allow  you  another  20  minutes 
to  complete  the  test  and  see  how  many  more  you  can 
get  ri.  ht.  If  you  wish  to  ,  o  back  and  change  any 
which  jou  have  already  answered,  you  may  do  so  in 
the  same  manner  as  described  before.   If  you  finish 
beiore  the  tixue  has  elapsed,  just  lay  the  test  aside 
and  wait  quietly  until  the  tests  are  collected.  Are 
there  any  questions?" 
{ If  none ,  say : ) 
"You  may  now  continue," 
(After  another  20  minutes,  say:) 
"Stop," 

ild  your  answer  sheet  in  your  booklet  and  turn 
it  so  that  the  page  with  your  name  is  on  top," 
(Collect  booklets  Immediately,  ) 
This  test  was  given  in  this  manner  during  the  1944-45 

term  to  a  group  of  574  tenth  grade  students  In  the  Wyandotte 

High  School  In  Kansas  City,  Kansas 

Grades  used  are  grades  earned  during  the  1944-45  term 

except  In  the  case  or  general  mathematics  In  which  case  they 

are  lor  the  preceding  term. 
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On  the  coll©;  o  level  the  test  was  given  to  the  ireshaan 
class  at  Kansas  State  College  in  the  rail  or  1943,  Grades 
used  i'or  this  part  or  the  study  were  those  earnoa  during  the 
first  semester  of  the  ly4i3-44  term. 


13 


III.  R 

In  the  statistical  treatment  of  the  data,  the  various 
{•roups  used  inoludad  all  students  who  took  the  subjects  or 
combinations  of  subjects  necessary  for  the  particular  calcula- 
tions involved.  For  the  most  important  pert  of  the  study  the 
group  used  consisted  of  112  boys  for  whom  grades  were  avail- 
able in  algebra,  general  mathematics,  and  mechanical  drawin  . 
Correlations  were  calculated  by  the  use  oi  Ayers '  formula, 
one  of  the  common  methods  of  obtaining  the  product-moaent 
coefficient  of  correlation  (1).  The  inter-correlations  for 
this  group  between  the  time-limit  scores,  the  work-limit 
scores,  the  Henmon-Keloon  Ikri  llij-ence  Test  scores,  and  grades 
In  general  mathematics,  clrebra,  and  mechanical  drawing  are 
shown  in  Table  I. 

Table  I.  Correlations  between  tine-limit  scores,  work-limit 
scores,  intelligence  test  scores,  and  grades. 


Alg.   Gen.  Math.   Mech.  Dr.   Int.  test 


Time-limit  scores    .239      .309         .232        .2 
work-limit  scores    .336      .361         .357        .443 
Int.  test  scores     .310      .  ,o5        .367 

It  will  be  noted  irom  this  table  that  in  each  of  thesa 

three  subjects  the  work-limit  scores  gave  higher  correlations 

than  did  the  time-limit  scores,  the  uifferences  being  .097  in 

algebra,  .052  in  general  mathematics,  and  .127  in  mechanical 

drawing.  It  will  also  be  noted  that  the  correlations  of  the 

intelligence  test  scores  with  the  subject  grades  exceeded 


14 


those  of  the  time-lixit  scores  in  algebra  and  in  mechanical 
drawing    but  were  lower  than  those  of  the  work-limit  scores 
except  in  mechanical  drawii    ,    in  which  they  were  practically 
identical,   the  difference   bei:  which  is  not  a   .  1   .  if- 

leant  difference.     Another  point  might  be  noted  here.     The 
work-limit  scores  correlated   .241  b*r  with  the  Intelligence 

test  scores  than  did  the  tine-liiult  scores. 

These  differences  in  correlation,   the  significance  of 
which  will  be  discusaod  later,   are  more  meaningful  when  they 
are  translated  into  percentaf  es  of  forecasting  efficiency. 
This  means  the  per  cent  greater  than  char.ee  by  which  one  is 
able  to  forecast  a  person's  success  in  a  subject  by  us' 
his  test  scores.     This  percent  is  fouiid  by  the  formula  100 (1-k) 


where  k=~^f  1-r2.     These  percentages  are  shown  in  Table  II. 

Table   II.     Percentages  of  forecasting  efficiency  of  time- 
limit  and  work-liinit  scores. 

Alg.        Pen.  Math.       Mech.   Dr.        Int.  .cores 

Time-limit  4.91  2.68J&  4.  :i, 

Work-limit  b.96%  6.r  .70  10.20JC 

Thus  the  forecasting  efficiency  of  the  tine-limit   scores  in 
predicting  success  in  algebrr.  is  seen  to  bo  2.92  per  cent  better 
than  chance.     For  the  work-limit  scores,    the  correspond!:, 
figure  is  5.96  per  cent,   or  a  little  more  than  double   the   tin-a- 
it figure.     For  general  mat heme tic a,    the   time-limit  scores 
give  a  forecasting  efficiency  of  4. 93  per  cent;  the  work-limit 
scores  give  6.70  per  cent.     In  mechanical  drawing  the  forecast- 
ing efficiencies  are:     tine-limit,   2.68  percent;  work-limit, 
6.70  per  cent.      In  forecasting  intelligence   test  scores,   the 
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efficiency  of  the  timo-liult  teat  is  4.01  per  cent;  that  of  the 

work-limit  teat  is  10.20  per  cent. 

Partial  correlations  were  calculated  by  the  formula 

i;,,=  r'»  ~  ,  r>3  Tzl,         for  both  work-limit  and  time -limit  scores 

with  each  of  the  three  subjects,  the  intelligence  scores  being 

partialed  out.  These  partial  correlations  are  shown  in  Table  III. 

Table  III.  Comparison  of  aero  order  r's  and  r'a  with  effect 
of  intelligence  teat  score a  partialed  out. 

Z]  Gen.  Math.  Mech.  Dr. 

^■ro  :.   >artlal  r   Zero  r  Partial  r   Zerc  r  Partial  r 

limlt     .  39  .162       .309     .262       .232     .138 

limit     .336     .233       .361     .286       .357     .233 
This  table  ahows  that  the  partialing  out  of  intelligence  aa 
measured  by  this  teat  reducea  the  correlations  of  both  work- 
limit  and  time-limit  scores  with  ftll  the  subjects,  but  that  the 
correlations  of  work-limit  scores  and  aubjects  atill  remain 
higher  than  the  corresponding  correlations  of  time-limit  acorea 
and  aubjects.  This  would  seem  to  indicate  that  intelligence 
sa  measured  by  the  Kenmon-^elson  Test  is  a  contributing  factor 
in  succeeding  on  the  Minnesota  Pa;    orm  Board  ToBt  on  either 
the  work-limit  or  the  time-limit  basis.  These  partial  coeffi- 
cients may  have  many  implications  but  from  the  standpoint  of 
this  study  the  point  to  be  noted  is  that  the  correlations  of 
work-limit  scores  with  each  subject  with  intelligence  partial- 
ed out  are  in  each  case  almoat  Identical  with  the  correlationa 
of  aero  order  tlme-iiniit  acorea  and  aubjecta  and  are  conaider- 
ably  higher  than  correlationa  of  time-limit  scores  with  lntell- 


16 


.ncc  partialed  out. 

Multiple  correlations  were  calculated  by  the  use  of  Kelley'a 


formula  H ,  (23) lj    TfA,  * V'f  "    - 2T,Z  r'3  r*3      |,   tola  formula  has 
the  advantage  over  the  one  involving  partial  signas  in  that  it 


makes  it  possible  to  calculate  mull     ;oaff iciente  directly 

from  the  ae:  o  ficleni.n.       ding  this  formula, 

Kelley  states,  "Sinco  it  is  the  same  as  the  equation  resulting 

from  the  solution  of  a  three  variable  problem  by  means  of 

determinants,  we  have  additional  proof  of  the  identity  of  the 

values  found  by  the  two  techniques." 

Table  IV  shows  the  multiple  correlations  obtained  by 

correlating  subject  grades  with  combined  time-limit  and  Intel- 

once  te.t  scores,  and  by  correlating  subject  grades  with 

combined  work-limit  and  i       nco  teat  scores. 

Table  IV.  Correlations  of  subject  gradva  with  tine-limit  and 
work-limit  scores  each  combined  with  intelllf ence 
test  scores. 


a  if. 

s 

i.  Hath. 

Kech.  Dr. 

Tine -limit 

.2<" 

.309 

.232 

Int.  score 

. 

.2. 

.367 

Both  in 

.351 

.388 

Work- 

.336 

•J 

.357 

Int.  score 

.310 

•24 

.367 

Both  In  lault. 

.3- 

.376 

.426 

It  can  be  noted  from  this  table  that  the  correlations  using 
work-limit  and  intelligence  scores  are  higher  than  those 
using  the  tine-limit  and  intelligence  te3t  scores.  However, 
it  is  also  evidont  from  this  table  thst,  due  to  the  lower 
correlation  of  time-limit  with  Intelligence  test  scores,  the 
differences  in  the  multiple  correlations  are  less  than  the 
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differences  In  aero  order  correlations.  In  algebra  the  differ- 
ence in  zero  order  correlations  la  .097  in  favor  of  the  work- 
limit;  when  used  in  multiple  correlation  with  intelligence 
•cores,  the  difference  la  only  .037.  In  general  mathematics 
the  difference  in  zero  order  correlations  la  .052  in  favor  of 
the  work-limit;  when  used  in  multiple  correlation  with  intelli- 
gence scores,  this  difference  If     sed  to  .025.   In  mechanical 
drawing  the  difference  In  zero  order  correlations  is  .127  in 
favor  of  the  work-limit;     i  usod  in  an     ■  correlation  with 
intelligence  acor€«s,  the  difference  la  only. 040. 

Before  any  conclusions  could  be  drawn  concernin      rel- 
ative tnorits  of  the  work-li^.it  method  over  the  time-limit 
method,  it  was  necoasary  to  determine  whether  or  not  these 
differences  in  correlations  are  actual  or  whether  they  might 
be  due  to  chance.  This  was  done  by  finding  the  standard  error 
of  the  difforonces  In  correlations  in  each  case  by  the  formula 

no  =~1  ^r?  +  K*     whero  C  =  -1-"-—  and  <T  1"r^   and  then 

by  dividing  tho  actual  dii'forences  by  the  respective  atandaxn. 
errors  of  alfferencea  to  net  the  significance  ratios,  1'heaa 
aignificance  ratioa  were  then  used  to  obtain  th*;  levels  of 
con       at   which  the  differences  are  significant.  These 
figurea  are  shown  for  the  differences  in  zero  order  ccrrelationa 
In  Table  V  and  for  the  differences  in  multiple  correlations  » 
In  Table  VI.   It  will  be  noted  from  these  tables  that  in  no 
case  does  the  level  of  confidence  approach  five  per  cent,  I 
minimum  acceptable  level  for  concluding  a  I     leant  dif- 
ference • 
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Table  V.  SignificEnce  of  differences  between  correlations 
with  grades  of  work-limit  scores  and  tiree-lirait 
scores. 

j:df'.      "7>           "  .  i'  .      .nee  a  l              \evol  of 
inr'a     °                  aio      that      confidence 
dif.  is  alp. 


ebrs     .097    .122       .V         79  21% 

Gen.  Math.   .052    .118       .         67  S3* 

Mech.  Dr.    .127    .121      1 .05        85 


Taole  VI.  Significance  oi   differences  between  multiple  cor- 
relations with  grades  of  work-Unit  and  intelli- 
gence scores  and  time-limit  and  intelligence  acoros. 


Dif. 

(P 

llg. 

CbMMMl    IB 

nevel    Of 

in  rfs 

VD 

Ratio 

100  that 

dif.   is  sig. 

confidence 

Algebra     ,057    .116       .32       63  37% 

Gen.  Math.   .025    .110       .2        59  11 

Mecb.  Dr.    .040    .111       .36        M  3v 

o.vbver,  as  pointed  out  by  Garrett  (5)  and  by  Llndquist 

(lu-b)  to  test  the  differences  between  correlation  coefficients 

by  the  lormula  gives  estimated  standard  errors 

that  are  larger  than  the  standard  errors  actually  are.  This 

aeans  that  the  chances  of  a  slgnll leant  difference  aro  actually 

greater  than  those  shown  by  Tables  V  and  VI.  It  is  impossible 

to  say  Just  how  much  greater  the  chances  are  than  those  shown 

by  applying  the  aforementioned  formula,  since  no  test  of  the 

significance  of  a  difference  in  correlation  coefficients  for 

this  situation  has  been  devised*  Lindquist  (10-b)  gives  the 


formula  <v  r\|-A.  fc*+A.  ^nc'is-z^  -y^7/a  ''  *3  -*,\  --£   -Sr^r,,^,? 


in  which  k  =~d 1-r*.  He  points  out,  however,  that  its  use- 
less is  impaired  by  the  xact  that  it  can  oe  used  only  upon 
the  assumptions  of  normality  ana  random  sampling.   Ihourh  it 
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would  bo  fallacious  to  assume  eith  .r  of  these  conditions  In 
this  study.  It  la  interesting  to  r.ote  what  results  the  appli- 
cation of  this  formula  producea  In  comparison  with  thoae  obtain- 
ed by  the  use  of  the  other  formula.   One  example  w;.s  choaen  and 
worked  through  to  show  this  comparison.  The  example  chosen  waa 
mechanical  drawing,  in  which,  as  shown  by  Table  V,  the  signifi- 
cance ratio  Obtained  waa  1.05  indicating  a  signif leant  difference 
at  the  15  per  cent  level  of  confidence.  When  the  formula  men- 
tioned by  Llndquist  waa  used,  the  significance  ratio  was  2.48 
which  indicates  a  significant  difference  at  better  than  the  one 
per  cent  level.  i>ue  to  the  limitations  on  this  iormula  already 
mentioned,  this  ootained  significance  ratio  is  probably  higher 
than  the  true  significance  ratio.  However,  the  1.05  found  by 
the  oth^r  formula  is  doubtless  too  low  and  the  differences  in 
correlation  coefficients  may  be  significant.   It  is  Impossible 
to  make  definite  conclusions  on  the  basis  of  the  evidence 
brought  out  in  this  study,  but  the  evidence  points  toward  better 
diagnostic  value  of  the  teat  for  mathematics  and  mechanical 
drawing  in  high  school  if  used  on  the  work-limit  basis. 

Mention  should  also  be  made  here  of  another  fact  brought 
out  In  this  study.  A  group  of  157  boys  who  had  taken  the  test 
by  both  methods  were  also  enrolled  In  general  shop.  This 
group  was  not  Included  in  the  previously  discussed  part  of  the 
study  for  two  reasons.  First,  there  were  so  few  of  these  boys 
who  had  also  taken  the  other  subjects  included;  and  second,  It 
is  doubtful  that  the  purely  mental  aspects  of  mechanical  abil- 
ity are  as  important  lectors  in  success  in  shop  work  as  they 
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•re  in  success  in  mathematics  and  mechanical  drawing.  However, 
correlations  of  both  time-limit  and  work-limit  scores  with  shop 
grades  wore  calculated  and  it  is  interesting  to  note  that  time- 
limit  scores  correlated  .319  with  shop  grades  while  work-limit 
scores  correlated  only  .275.  This  is  a  difference  of  .044  in 
favor  or  the  time-limit  method*  The  standard  error  of  this 
difference  obtained  by  the  formula  0J  ="~^  ^*  +  #£     i8  ,io3  which 
yields  a  significance  ratio  of  .43.  This  means  that  there  are 
66  chances  in  100  that  the  difference  is  significant.  Since 
this  significance  can  be  regarded  only  at  the  34  per  cent  level 
of  confidence,  it  is  not  a  very  emphatic  indicator,  but  might 
suggest  that  speed  of  recognition  of  space  relationships  is 
more  indicative  of  ability  in  work  such  as  that  done  in  shop 
courses,  while  level  of  attainment  of  recognising  space  rela- 
tionships is  more  indicative  of  success  in  subjects  that  require 
the  use  of  the  mental  processes  involved  in  mathematics  and 
mechanical  drawing  courses. 

Since  the  work-limit  scores  gave  hi*  her  correlations  with 
all  subjects  used  except  shop,  and  since  the  reliability  of 
the  test  on  a  work-limit  basis  had  not  been  calculated,  that 
was  also  done  to  see  whether  or  not  the  increased  validity  for 
this  group  could  be  accounted  for  by  increased  reliability. 
The  reliability  of  the  test  on  tho  time-limit  basis  as  given 
oy  the  authors  in  the  manual  of  directions  is  .86.  They  do 
not  state  the  method  used  in  obtaining  this  figure.   In  this 
study  the  split-half  method  was  used  as  the  test  was  given 
only  once.  The  r's  ior  the  half -scores  were  corrected  by  the 


21 


Spearman-3rown  formula  to  give  correct  r's  for  the  whole  test. 
The  reliability  on  the  time-limit  was  found  to  be  .844,  almost 
identical  with  that  stated  by  the  authors;  and  on  the  work- 
limit  basis  it  was  .834.  However,  as  Guilford  (6)  points  out, 
the  reliability  coefficient  of  a  time-limit  test  is  augmented 
to  the  extent  that  speed  is  important  in  determining  the  score. 
Therefore,  when  two  tests  are  compared  for  reliability  and  one 
is  a  time-limit  test  and  the  other  a  work-limit  test;  if  the 
reliability  coefficients  are  equal,  the  work-limit  test  would 
actually  be  more  reliable.  Since  there  is  no  known  correction 
for  this  discrepancy  between  apparent  and  actual  reliability 
coefficients  for  time-limit  tests,  it  can  only  be  said  that 
the  work-limit  basis  has  some  advantage  over  the  time-limit 
basis  from  the  standpoint  of  reliability.  This  may  explain 
in  part  the  increases  in  validity  of  the  test  when  given  on 
a  work-limit  basis. 

As  was  mentioned  in  the  introduction,  this  study  was 
primarily  concerned  with  the  use  of  the  Minnesota  Paper  Form 
Board  Test  at  the  high-school  level.   However,  a  brief  summary 
of  the  correlations  of  both  time-limit  and  work-limit  scores 
with  college  grades  and  other  tests  is  shown  in  Table  VII. 
It  can  be  noted  rrom  this  table  that  in  all  cases  In  which  the 
correlations  of  time-limit  with  grades  exceeded  those  of  work- 
limit  with  grades  the  groups  Involved  were  very  small,  the 
largest  being  only  45.   It  should  also  be  pointed  out  that 
due  to  the  national  emergency  In  the  fall  of  1943,  there  were 
very  few  men  enrolled  as  freshmen  in  college;  hence,  all 


88 


groups  used  here  Include  both  men  and  women.  In  view  of  these 

facts  and  also  the  fact  that  all  differences  In  correlations 

are  small,  except  in  the  case  of  general  psychology,  one 

should  not  depend  too  heavily  upon  these  results.  However, 

these  figures  seem  to  indicate  that  the  work-limit  might  be 

a  better  predictor  at  the  college  level,  especially  since  it 

correlates  .lid6  hirher  than  the  tliae-linlt  with  the  Kansas 

Math  Test, 

Table  VII.  Correlations  of  time-limit  and  work-lirr.it  scores 
with  college  grades  and  other  tests. 


Subject 

No.  of 

r,  for 

rz  for 

*a  -r, 

or  tost 

cases 

timo- 
limit 

wo; 
limit 

"■•    / 

Solid  geosu 

20 

•234 

.215 

-.019 

Chem. 

36 

.  .13 

.098 

-.115 

Chom.  I 

184 

51 

.290 

.039 

Gen.  Psychol, 

91 

•  25*0 

.518 

.228 

Engg.  Dr. 

45 

•490 

.377 

-.113 

5  Hr.  Alg, 

50 

.200 

.204 

.  04 

3  lir.  Alg. 

45 

.163 

.325 

.162 

Tri'  . 

30 

.345 

.218 

-.127 

m.  Desl* 

o-J 

•  2G1 

•329 

•068 

Rhet.  I 

235 

•  254 

.258 

•004 

Kan.  Math  Test 

280 

.167 

•293 

.126 

Gen.  Int.  Tost 

275 

.345 

.401 

.056 
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IV.   CO 

1.  The  reliability  of  the  teat  la  not  impaired  by  using  the 
work-limit  method, 

2.  For  fcb«  rroupa  studied  at  the  high-school  level,  the 
validity  was  improved  so  far  as  mathematics  and  mechanical 
drawing  were  concerned  by  using  the  work-limit  method  but 
was  decreased  when  shop  grades  were  used  as  the  criterion, 

3.  Since  the  differences  in  validity  were  not  sufficiently 
large  to  assure  at  a  high  level  of  confidence  that  the 
differences  found  in  the  samples  tested  would  exist  in 
other  samples,  neither  method  should  be  adopted  exclusively 
on  the  basis  of  this  study.  However,  the  indications  are 
that  if  the  purely  mental  aspects  of  perception  of  spatial 
relationship  are  to  be  taken  as  the  criteria,  the  work- 
limit  method  is  probably  the  more  valid  of  the  two, 

4.  Since  the  work-limit  method  is  as  reliable  as  the  time- 
liicit  method  and  appears  to  be  more  valid  for  measuring 
mental  aspects  of  mechanical  ability,  and  since  the  test 
is  likely  to  become  wore  and  more  widely  used  by  inexper- 
ienced testers  who  are  likely  to  do  Inaccurate  timing,  the 
work-limit  method  should  receive  serious  consideration, 

6.  Since  present  norms  are  based  on  time-limit  scores,  new 
norms  must  be  developed  if  the  work-limit  method  is  to 
be  used, 

6,  This  study  has  brought  up  other  related  problems  which 
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need  to  be  investigated,  some  of  which  are: 

(a).  Time-limit  vs.  work-limit  methods  on  other 

commonly  used  teste, 
(b).  The  possible  combining  of  both  time-limit  and 

work-limit  scores  for  predictive  purposes, 
(c).  Further  checking  of  the  work-limit  method  on 
this  particular  test. 
On  the  college  level,  the  evidence  points  toward  a  slightly 

her  validity  for  the  work-limit  basis;  however,  further 
research  should  be  done  before  any  conclusions  are  drawn. 
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